{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/NeuromatchAcademy/course-content/blob/master/tutorials/W3D1_RealNeurons/student/W3D1_Tutorial3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "# Neuromatch Academy: Week 2, Day 3, Tutorial 3\n",
    "# Real Neurons: Synaptic transmission - Models of static and dynamic synapses\n",
    "__Content creators:__ Qinglong Gu, Songtin Li, John Murray, Richard Naud, Arvind Kumar\n",
    "\n",
    "__Content reviewers:__  Maryam Vaziri-Pashkam, Ella Batty, Lorenzo Fontolan, Richard Gao, Matthew Krause, Spiros Chavlis, Michael Waskom"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Tutorial Objectives\n",
    "Synapses connect neurons into neural networks or circuits. Specialized electrical synapses make direct, physical connections between neurons. In this tutorial, however, we will focus on **chemical synapses**, which are more common in the brain. These synapses do not physically join neurons. Instead, a spike in the presynaptic cell causes a chemical, or neurotransmitter, to be released into a small space between the neurons called the synaptic cleft. Once the chemical diffuses across that space, it changes the pearmeability of the postsynaptic membrane, which may result in a positive or negative change in the membrane voltage.\n",
    "\n",
    "In this tutorial, we will model chemical synaptic transmission and study some interesting effects produced by **static synapses** and **dynamic synapses**.\n",
    "\n",
    "First, we will start by writing code to simulate static synapses -- whose weight is always fixed. \n",
    "Next, we will extend the model and model **dynamic synapses** -- whose synaptic strength is dependent on the recent spike history: synapses can either progressively increase or decrease the size of their effects on the post-synaptic neuron, based on the recent firing rate of its presynaptic partners. This feature of synapses in the brain is called **Short-Term Plasticity** and causes synapses to undergo *Facilitation* or *Depression*. \n",
    "\n",
    "Our goals for this tutorial are to:\n",
    "\n",
    "- simulate static synapses and study how excitation and inhibition affect the patterns in the neurons' spiking output\n",
    "- define mean- or fluctuation-driven regimes\n",
    "- simulate short-term dynamics of synapses (facilitation and depression)\n",
    "- study how a change in pre-synaptic firing history affects the synaptic weights (i.e., PSP amplitude)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 517
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:45.429173Z",
     "iopub.status.busy": "2021-05-25T01:13:45.428529Z",
     "iopub.status.idle": "2021-05-25T01:13:45.493643Z",
     "shell.execute_reply": "2021-05-25T01:13:45.494093Z"
    },
    "outputId": "e366c141-8c0c-4ff2-f1f1-1ae655b93e6e"
   },
   "outputs": [],
   "source": [
    "#@title Video 1: Static and dynamic synapses\n",
    "from IPython.display import YouTubeVideo\n",
    "video = YouTubeVideo(id='Hbz2lj2AO_0', width=854, height=480, fs=1)\n",
    "print(\"Video available at https://youtube.com/watch?v=\" + video.id)\n",
    "video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Setup\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:45.498194Z",
     "iopub.status.busy": "2021-05-25T01:13:45.497651Z",
     "iopub.status.idle": "2021-05-25T01:13:45.802261Z",
     "shell.execute_reply": "2021-05-25T01:13:45.801608Z"
    }
   },
   "outputs": [],
   "source": [
    "# Import libraries\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:45.807424Z",
     "iopub.status.busy": "2021-05-25T01:13:45.806865Z",
     "iopub.status.idle": "2021-05-25T01:13:45.888175Z",
     "shell.execute_reply": "2021-05-25T01:13:45.887651Z"
    }
   },
   "outputs": [],
   "source": [
    "# @title Figure Settings\n",
    "import ipywidgets as widgets       # interactive display\n",
    "%config InlineBackend.figure_format='retina'\n",
    "# use NMA plot style\n",
    "plt.style.use(\"https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle\")\n",
    "my_layout = widgets.Layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:45.913215Z",
     "iopub.status.busy": "2021-05-25T01:13:45.896177Z",
     "iopub.status.idle": "2021-05-25T01:13:45.916355Z",
     "shell.execute_reply": "2021-05-25T01:13:45.915866Z"
    }
   },
   "outputs": [],
   "source": [
    "# @title Helper functions\n",
    "\n",
    "\n",
    "def my_GWN(pars, mu, sig, myseed=False):\n",
    "  \"\"\"\n",
    "  Args:\n",
    "    pars       : parameter dictionary\n",
    "    mu         : noise baseline (mean)\n",
    "    sig        : noise amplitute (standard deviation)\n",
    "    myseed     : random seed. int or boolean\n",
    "                  the same seed will give the same random number sequence\n",
    "\n",
    "  Returns:\n",
    "    I          : Gaussian White Noise (GWN) input\n",
    "  \"\"\"\n",
    "\n",
    "  # Retrieve simulation parameters\n",
    "  dt, range_t = pars['dt'], pars['range_t']\n",
    "  Lt = range_t.size\n",
    "\n",
    "  # set random seed\n",
    "  # you can fix the seed of the random number generator so that the results\n",
    "  # are reliable. However, when you want to generate multiple realizations\n",
    "  # make sure that you change the seed for each new realization.\n",
    "  if myseed:\n",
    "      np.random.seed(seed=myseed)\n",
    "  else:\n",
    "      np.random.seed()\n",
    "\n",
    "  # generate GWN\n",
    "  # we divide here by 1000 to convert units to seconds.\n",
    "  I_GWN = mu + sig * np.random.randn(Lt) / np.sqrt(dt / 1000.)\n",
    "\n",
    "  return I_GWN\n",
    "\n",
    "\n",
    "def Poisson_generator(pars, rate, n, myseed=False):\n",
    "  \"\"\"\n",
    "  Generates poisson trains\n",
    "\n",
    "  Args:\n",
    "    pars       : parameter dictionary\n",
    "    rate       : noise amplitute [Hz]\n",
    "    n          : number of Poisson trains\n",
    "    myseed     : random seed. int or boolean\n",
    "\n",
    "  Returns:\n",
    "    pre_spike_train : spike train matrix, ith row represents whether\n",
    "                      there is a spike in ith spike train over time\n",
    "                      (1 if spike, 0 otherwise)\n",
    "  \"\"\"\n",
    "\n",
    "  # Retrieve simulation parameters\n",
    "  dt, range_t = pars['dt'], pars['range_t']\n",
    "  Lt = range_t.size\n",
    "\n",
    "  # set random seed\n",
    "  if myseed:\n",
    "      np.random.seed(seed=myseed)\n",
    "  else:\n",
    "      np.random.seed()\n",
    "\n",
    "  # generate uniformly distributed random variables\n",
    "  u_rand = np.random.rand(n, Lt)\n",
    "\n",
    "  # generate Poisson train\n",
    "  poisson_train = 1. * (u_rand < rate * (dt / 1000.))\n",
    "\n",
    "  return poisson_train\n",
    "\n",
    "\n",
    "def default_pars(**kwargs):\n",
    "  pars = {}\n",
    "\n",
    "  ### typical neuron parameters###\n",
    "  pars['V_th'] = -55.    # spike threshold [mV]\n",
    "  pars['V_reset'] = -75. # reset potential [mV]\n",
    "  pars['tau_m'] = 10.    # membrane time constant [ms]\n",
    "  pars['g_L'] = 10.      # leak conductance [nS]\n",
    "  pars['V_init'] = -65.  # initial potential [mV]\n",
    "  pars['E_L'] = -75.     # leak reversal potential [mV]\n",
    "  pars['tref'] = 2.      # refractory time (ms)\n",
    "\n",
    "  ### simulation parameters ###\n",
    "  pars['T'] = 400.  # Total duration of simulation [ms]\n",
    "  pars['dt'] = .1   # Simulation time step [ms]\n",
    "\n",
    "  ### external parameters if any ###\n",
    "  for k in kwargs:\n",
    "    pars[k] = kwargs[k]\n",
    "\n",
    "  pars['range_t'] = np.arange(0, pars['T'], pars['dt'])  # Vector of discretized time points [ms]\n",
    "\n",
    "  return pars\n",
    "\n",
    "\n",
    "def my_illus_LIFSYN(pars, v_fmp, v):\n",
    "  \"\"\"\n",
    "  Illustartion of FMP and membrane voltage\n",
    "\n",
    "  Args:\n",
    "    pars  : parameters dictionary\n",
    "    v_fmp : free membrane potential, mV\n",
    "    v     : membrane voltage, mV\n",
    "\n",
    "  Returns:\n",
    "    plot of membrane voltage and FMP, alongside with the spiking threshold\n",
    "    and the mean FMP (dashed lines)\n",
    "  \"\"\"\n",
    "\n",
    "  plt.figure(figsize=(14, 5))\n",
    "  plt.plot(pars['range_t'], v_fmp, 'r', lw=1.,\n",
    "           label='Free mem. pot.', zorder=2)\n",
    "  plt.plot(pars['range_t'], v, 'b', lw=1.,\n",
    "           label='True mem. pot', zorder=1, alpha=0.7)\n",
    "  plt.axhline(-55, 0, 1, color='k', lw=2., ls='--',\n",
    "              label='Spike Threshold', zorder=1)\n",
    "  plt.axhline(np.mean(v_fmp), 0, 1, color='r', lw=2., ls='--',\n",
    "              label='Mean Free Mem. Pot.', zorder=1)\n",
    "  plt.xlabel('Time (ms)')\n",
    "  plt.ylabel('V (mV)')\n",
    "  plt.legend(loc=[1.02, 0.68])\n",
    "  plt.show()\n",
    "\n",
    "\n",
    "def my_illus_STD(Poi_or_reg=False, rate=20., U0=0.5,\n",
    "                 tau_d=100., tau_f=50., plot_out=True):\n",
    "  \"\"\"\n",
    "   Only for one presynaptic train\n",
    "\n",
    "  Args:\n",
    "    Poi_or_reg : Poisson or regular input spiking trains\n",
    "    rate       : Rate of input spikes, Hz\n",
    "    U0         : synaptic release probability at rest\n",
    "    tau_d      : synaptic depression time constant of x [ms]\n",
    "    tau_f      : synaptic facilitation time constantr of u [ms]\n",
    "    plot_out   : whether ot not to plot, True or False\n",
    "\n",
    "  Returns:\n",
    "    Nothing.\n",
    "  \"\"\"\n",
    "\n",
    "  T_simu = 10.0 * 1000 / (1.0 * rate)  # 10 spikes in the time window\n",
    "  pars = default_pars(T=T_simu)\n",
    "  dt = pars['dt']\n",
    "\n",
    "  if Poi_or_reg:\n",
    "    # Poisson type spike train\n",
    "    pre_spike_train = Poisson_generator(pars, rate, n=1)\n",
    "    pre_spike_train = pre_spike_train.sum(axis=0)\n",
    "  else:\n",
    "    # Regular firing rate\n",
    "    isi_num = int((1e3/rate)/dt)  # number of dt\n",
    "    pre_spike_train = np.zeros(len(pars['range_t']))\n",
    "    pre_spike_train[::isi_num] = 1.\n",
    "\n",
    "  u, R, g = dynamic_syn(g_bar=1.2, tau_syn=5., U0=U0,\n",
    "                        tau_d=tau_d, tau_f=tau_f,\n",
    "                        pre_spike_train=pre_spike_train,\n",
    "                        dt=pars['dt'])\n",
    "\n",
    "  if plot_out:\n",
    "    plt.figure(figsize=(12, 6))\n",
    "\n",
    "    plt.subplot(221)\n",
    "    plt.plot(pars['range_t'], R, 'b', label='R')\n",
    "    plt.plot(pars['range_t'], u, 'r', label='u')\n",
    "    plt.legend(loc='best')\n",
    "    plt.xlim((0, pars['T']))\n",
    "    plt.ylabel(r'$R$ or $u$ (a.u)')\n",
    "    plt.subplot(223)\n",
    "    spT = pre_spike_train > 0\n",
    "    t_sp = pars['range_t'][spT]  #spike times\n",
    "    plt.plot(t_sp, 0. * np.ones(len(t_sp)), 'k|', ms=18, markeredgewidth=2)\n",
    "    plt.xlabel('Time (ms)');\n",
    "    plt.xlim((0, pars['T']))\n",
    "    plt.yticks([])\n",
    "    plt.title('Presynaptic spikes')\n",
    "\n",
    "    plt.subplot(122)\n",
    "    plt.plot(pars['range_t'], g, 'r', label='STP synapse')\n",
    "    plt.xlabel('Time (ms)')\n",
    "    plt.ylabel('g (nS)')\n",
    "    plt.xlim((0, pars['T']))\n",
    "\n",
    "    plt.tight_layout()\n",
    "\n",
    "  if not Poi_or_reg:\n",
    "    return g[isi_num], g[9*isi_num]\n",
    "\n",
    "\n",
    "def plot_volt_trace(pars, v, sp):\n",
    "  \"\"\"\n",
    "  Plot trajetory of membrane potential for a single neuron\n",
    "\n",
    "  Args:\n",
    "    pars   : parameter dictionary\n",
    "    v      : volt trajetory\n",
    "    sp     : spike train\n",
    "\n",
    "  Returns:\n",
    "    figure of the membrane potential trajetory for a single neuron\n",
    "  \"\"\"\n",
    "\n",
    "  V_th = pars['V_th']\n",
    "  dt = pars['dt']\n",
    "  if sp.size:\n",
    "      sp_num = (sp/dt).astype(int) - 1\n",
    "      v[sp_num] += 10\n",
    "\n",
    "  plt.plot(pars['range_t'], v, 'b')\n",
    "  plt.axhline(V_th, 0, 1, color='k', ls='--', lw=1.)\n",
    "  plt.xlabel('Time (ms)')\n",
    "  plt.ylabel('V (mV)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "In the `Helper Function`:\n",
    "\n",
    "- Gaussian white noise generator: `my_GWN(pars, mu, sig, myseed=False)`\n",
    "- Poissonian spike train generator: `Poisson_generator(pars, rate, n, myseed=False)`\n",
    "- default parameter function (as before) and other plotting utilities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Section 1: Static synapses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Section 1.1: Simulate synaptic conductance dynamics\n",
    "\n",
    "Synaptic input _in vivo_ consists of a mixture of **excitatory** neurotransmitters, which depolarizes the cell and drives it towards spike threshold, and **inhibitory** neurotransmitters that hyperpolarize it, driving it away from spike threshold. These chemicals cause specific ion channels on the postsynaptic neuron to open, resulting in a change in that neuron's conductance and, therefore, the flow of current in or out of the cell.\n",
    "\n",
    "This process can be modelled by assuming that the presynaptic neuron's spiking activity produces transient changes in the postsynaptic neuron's conductance ($g_{\\rm syn}(t)$). Typically, the conductance transient is modeled as an exponential function. \n",
    "\n",
    "Such conductance transients can be generated using a simple ordinary differential equation (ODE):\n",
    "\n",
    "\\\\\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\frac{dg_{\\rm syn}(t)}{dt} &=& \\bar{g}_{\\rm syn} \\sum_k \\delta(t-t_k) -g_{\\rm syn}(t)/\\tau_{\\rm syn}\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\\\\n",
    "\n",
    "where $\\bar{g}_{\\rm syn}$ (often referred to as synaptic weight) is the maximum conductance elicited by each incoming spike, and $\\tau_{\\rm syn}$ is the synaptic time constant. Note that the summation runs over all spikes received by the neuron at time $t_k$.\n",
    "\n",
    "Ohm's law allows us to convert conductance changes to the current as:\n",
    "\n",
    "\\\\\n",
    "\n",
    "\\begin{align}\n",
    "I_{\\rm syn}(t) = g_{\\rm syn}(t)(V(t)-E_{\\rm syn})  \\\\\n",
    "\\end{align}\n",
    "\n",
    "\\\\\n",
    "\n",
    "The reversal potential $E_{\\rm syn}$ determines the direction of current flow and the excitatory or inhibitory nature of the synapse. \n",
    "\n",
    "**Thus, incoming spikes are filtered by an exponential-shaped kernel, effectively low-pass filtering the input. In other words, synaptic input is not white noise, but it is, in fact, colored noise, where the color (spectrum) of the noise is determined by the synaptic time constants of both excitatory and inhibitory synapses.**\n",
    "\n",
    "In a neuronal network, the total synaptic input current $I_{\\rm syn}$ is the sum of both excitatory and inhibitory inputs. Assuming the total excitatory and inhibitory conductances received at time $t$ are $g_E(t)$ and $g_I(t)$, and their corresponding reversal potentials are $E_E$ and $E_I$, respectively, then the total synaptic current can be described as: \n",
    "\n",
    "\\\\\n",
    "\n",
    "\\begin{align}\n",
    "I_{\\rm syn}(V(t),t) = -g_E(t) (V-E_E) - g_I(t) (V-E_I)\n",
    "\\end{align}\n",
    "\n",
    "\\\\\n",
    "\n",
    "Accordingly, the membrane potential dynamics of the LIF neuron under synaptic current drive become:\n",
    "\n",
    "\\\\\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\tau_m\\frac{dV(t)}{dt} = -(V(t)-E_L) - \\frac{g_E(t)}{g_L} (V(t)-E_E) - \\frac{g_I(t)}{g_L} (V(t)-E_I) + \\frac{I_{\\rm inj}}{g_L}\\quad (2)\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\\\\n",
    "\n",
    "$I_{\\rm inj}$ is an external current injected in the neuron, which is under experimental control; it can be GWN, DC, or anything else.\n",
    "\n",
    "We will use Eq. (2) to simulate the conductance-based LIF neuron model below.\n",
    "\n",
    "In the previous tutorials, we saw how the output of a single neuron (spike count/rate and spike time irregularity) changes when we stimulate the neuron with DC and GWN, respectively. Now, we are in a position to study how the neuron behaves when it is bombarded with both excitatory and inhibitory spikes trains -- as happens *in vivo*.\n",
    "\n",
    "What kind of input is a neuron receiving? When we do not know, we chose the simplest option. The simplest model of input spikes is given when every input spike arrives independently of other spikes, i.e., we assume that the input is Poissonian."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Section 1.2: Simulate LIF neuron with conductance-based synapses\n",
    "\n",
    "We are now ready to simulate a LIF neuron with conductance-based synaptic inputs! The following code defines the LIF neuron with synaptic input modeled as conductance transients."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 354
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:45.929203Z",
     "iopub.status.busy": "2021-05-25T01:13:45.919301Z",
     "iopub.status.idle": "2021-05-25T01:13:45.933675Z",
     "shell.execute_reply": "2021-05-25T01:13:45.933225Z"
    },
    "outputId": "3b7a4aec-5c56-4796-bada-7362157337d2"
   },
   "outputs": [],
   "source": [
    "# @markdown Execute this cell to get a function for conductance-based LIF neuron (run_LIF_cond)\n",
    "\n",
    "def run_LIF_cond(pars, I_inj, pre_spike_train_ex, pre_spike_train_in):\n",
    "  \"\"\"\n",
    "  Conductance-based LIF dynamics\n",
    "\n",
    "  Args:\n",
    "    pars               : parameter dictionary\n",
    "    I_inj              : injected current [pA]. The injected current here\n",
    "                         can be a value or an array\n",
    "    pre_spike_train_ex : spike train input from presynaptic excitatory neuron\n",
    "    pre_spike_train_in : spike train input from presynaptic inhibitory neuron\n",
    "\n",
    "  Returns:\n",
    "    rec_spikes         : spike times\n",
    "    rec_v              : mebrane potential\n",
    "    gE                 : postsynaptic excitatory conductance\n",
    "    gI                 : postsynaptic inhibitory conductance\n",
    "\n",
    "  \"\"\"\n",
    "\n",
    "  # Retrieve parameters\n",
    "  V_th, V_reset = pars['V_th'], pars['V_reset']\n",
    "  tau_m, g_L = pars['tau_m'], pars['g_L']\n",
    "  V_init, E_L = pars['V_init'], pars['E_L']\n",
    "  gE_bar, gI_bar = pars['gE_bar'], pars['gI_bar']\n",
    "  VE, VI = pars['VE'], pars['VI']\n",
    "  tau_syn_E, tau_syn_I = pars['tau_syn_E'], pars['tau_syn_I']\n",
    "  tref = pars['tref']\n",
    "  dt, range_t = pars['dt'], pars['range_t']\n",
    "  Lt = range_t.size\n",
    "\n",
    "  # Initialize\n",
    "  tr = 0.\n",
    "  v = np.zeros(Lt)\n",
    "  v[0] = V_init\n",
    "  gE = np.zeros(Lt)\n",
    "  gI = np.zeros(Lt)\n",
    "  Iinj = I_inj * np.ones(Lt)  # ensure Iinj has length Lt\n",
    "\n",
    "  if pre_spike_train_ex.max() == 0:\n",
    "    pre_spike_train_ex_total = np.zeros(Lt)\n",
    "  else:\n",
    "    pre_spike_train_ex_total = pre_spike_train_ex.sum(axis=0) * np.ones(Lt)\n",
    "\n",
    "  if pre_spike_train_in.max() == 0:\n",
    "    pre_spike_train_in_total = np.zeros(Lt)\n",
    "  else:\n",
    "    pre_spike_train_in_total = pre_spike_train_in.sum(axis=0) * np.ones(Lt)\n",
    "\n",
    "  # simulation\n",
    "  rec_spikes = []  # recording spike times\n",
    "  for it in range(Lt - 1):\n",
    "    if tr > 0:\n",
    "      v[it] = V_reset\n",
    "      tr = tr - 1\n",
    "    elif v[it] >= V_th:  # reset voltage and record spike event\n",
    "      rec_spikes.append(it)\n",
    "      v[it] = V_reset\n",
    "      tr = tref / dt\n",
    "\n",
    "    # update the synaptic conductance\n",
    "    gE[it + 1] = gE[it] - (dt / tau_syn_E) * gE[it] + gE_bar * pre_spike_train_ex_total[it + 1]\n",
    "    gI[it + 1] = gI[it] - (dt / tau_syn_I) * gI[it] + gI_bar * pre_spike_train_in_total[it + 1]\n",
    "\n",
    "    # calculate the increment of the membrane potential\n",
    "    dv = (dt / tau_m) * (-(v[it] - E_L) \\\n",
    "                         - (gE[it + 1] / g_L) * (v[it] - VE) \\\n",
    "                         - (gI[it + 1] / g_L) * (v[it] - VI) + Iinj[it] / g_L)\n",
    "\n",
    "    # update membrane potential\n",
    "    v[it+1] = v[it] + dv\n",
    "\n",
    "  rec_spikes = np.array(rec_spikes) * dt\n",
    "\n",
    "  return v, rec_spikes, gE, gI\n",
    "\n",
    "\n",
    "print(help(run_LIF_cond))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "### Exercise 1: Measure the mean free membrane potential\n",
    "\n",
    "Let's simulate the conductance-based LIF neuron with presynaptic spike trains generated by a `Poisson_generator` with rate 10 Hz for both excitatory and inhibitory inputs. Here, we choose 80 excitatory presynaptic spike trains and 20 inhibitory ones.\n",
    "\n",
    "Previously, we've already learned that $CV_{\\rm ISI}$ can describe the irregularity of the output spike pattern. Now, we will introduce a new descriptor of the neuron membrane, i.e., the  **Free Membrane Potential (FMP)** -- the membrane potential of the neuron when its spike threshold is removed. \n",
    "\n",
    "Although this is completely artificial, calculating this quantity allows us to get an idea of how strong the input is. We are mostly interested in knowing the mean and standard deviation (std.) of the FMP. In the exercise, you can visualize the FMP and membrane voltage with spike threshold."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:45.940174Z",
     "iopub.status.busy": "2021-05-25T01:13:45.939639Z",
     "iopub.status.idle": "2021-05-25T01:13:45.999280Z",
     "shell.execute_reply": "2021-05-25T01:13:45.998751Z"
    }
   },
   "outputs": [],
   "source": [
    "# To complete the exercise, uncomment the code and fill the missing parts (...)\n",
    "pars = default_pars(T=1000.)\n",
    "# Add parameters\n",
    "pars['gE_bar'] = 2.4    # [nS]\n",
    "pars['VE'] = 0.         # [mV] excitatory reversal potential\n",
    "pars['tau_syn_E'] = 2.  # [ms]\n",
    "pars['gI_bar'] = 2.4    # [nS]\n",
    "pars['VI'] = -80.       # [mV] inhibitory reversal potential\n",
    "pars['tau_syn_I'] = 5.  # [ms]\n",
    "\n",
    "# generate presynaptic spike trains\n",
    "pre_spike_train_ex = Poisson_generator(pars, rate=10, n=80)\n",
    "pre_spike_train_in = Poisson_generator(pars, rate=10, n=20)\n",
    "\n",
    "# simulate conductance-based LIF model\n",
    "v, rec_spikes, gE, gI = run_LIF_cond(pars, 0, pre_spike_train_ex,\n",
    "                                     pre_spike_train_in)\n",
    "dt, range_t = pars['dt'], pars['range_t']\n",
    "if rec_spikes.size:\n",
    "  sp_num = (rec_spikes / dt).astype(int) - 1\n",
    "  v[sp_num] = 10   # draw nicer spikes\n",
    "\n",
    "####################################################################\n",
    "## TODO for students: measure the free membrane potential\n",
    "# In order to measure the free membrane potential, first,\n",
    "# you should prevent the firing of the LIF neuron\n",
    "# How to prevent a LIF neuron from firing? Increse the threshold pars['V_th'].\n",
    "####################################################################\n",
    "# Change the threshold\n",
    "# pars['V_th'] = ...\n",
    "# Calculate FMP\n",
    "# v_fmp, _, _, _ = run_LIF_cond(pars, ..., ..., ...)\n",
    "\n",
    "# uncomment when you have filled the exercise\n",
    "# my_illus_LIFSYN(pars, v_fmp, v)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 361
    },
    "colab_type": "text",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:46.006686Z",
     "iopub.status.busy": "2021-05-25T01:13:46.006116Z",
     "iopub.status.idle": "2021-05-25T01:13:46.482066Z",
     "shell.execute_reply": "2021-05-25T01:13:46.481551Z"
    },
    "outputId": "fcbf99e3-d44a-427f-a7d4-ff18f37388f6"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_RealNeurons/solutions/W2D3_Tutorial3_Solution_789f00e0.py)\n",
    "\n",
    "*Example output:*\n",
    "\n",
    "<img alt='Solution hint' align='left' width=991 height=344 src=https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/tutorials/W2D3_RealNeurons/static/W2D3_Tutorial3_Solution_789f00e0_0.png>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "### Interactive Demo: Conductance-based LIF Explorer with different E/I input\n",
    "\n",
    "In the following, we can investigate how varying the ratio of excitatory to inhibitory inputs changes the firing rate and the spike time regularity (see the output text). \n",
    "\n",
    "To change both the excitatory and inhibitory inputs, we will vary their firing rates. *However, if you wish, you can vary the strength and/or the number of these connections as well.*  \n",
    "\n",
    "Pay close attention to the mean free membrane potential (red dotted line) and its location with respect to the spike threshold (black dotted line). Try to develop a heuristic about the mean of the FMP and spike time irregularity ($CV_{\\rm ISI}$)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 817,
     "referenced_widgets": [
      "5cbacda71f45462f905ea9ddeb7e4a99",
      "6c5df7487a684471bd97bfa2e45db98b",
      "f499c0919126419ba6dbdccb0f11644e",
      "8e25495ac85242b9b2d0d0baf921cd0d",
      "6652f09e18c349139e8355b896a11518",
      "e1b1734c02cb4dfc9ceb6ea035aafe36",
      "f8dc53b40b774c9aa15b267cf1aed169",
      "629d24046f6f4fe68fd4108aad48ca14",
      "13e098e81f7b4b1bbae818e5565f721d"
     ]
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:46.519091Z",
     "iopub.status.busy": "2021-05-25T01:13:46.495331Z",
     "iopub.status.idle": "2021-05-25T01:13:47.585020Z",
     "shell.execute_reply": "2021-05-25T01:13:47.584532Z"
    },
    "outputId": "972c0dae-f8f1-481d-96b7-f6c27f9e30c0"
   },
   "outputs": [],
   "source": [
    "# @title\n",
    "\n",
    "# @markdown Make sure you execute this cell to enable the widget!\n",
    "\n",
    "my_layout.width = '450px'\n",
    "@widgets.interact(\n",
    "    inh_rate=widgets.FloatSlider(20., min=10., max=60., step=5.,\n",
    "                                 layout=my_layout),\n",
    "    exc_rate=widgets.FloatSlider(10., min=2., max=20., step=2.,\n",
    "                                 layout=my_layout)\n",
    ")\n",
    "\n",
    "\n",
    "def EI_isi_regularity(exc_rate, inh_rate):\n",
    "\n",
    "  pars = default_pars(T=1000.)\n",
    "  # Add parameters\n",
    "  pars['gE_bar'] = 3.     # [nS]\n",
    "  pars['VE'] = 0.         # [mV] excitatory reversal potential\n",
    "  pars['tau_syn_E'] = 2.  # [ms]\n",
    "  pars['gI_bar'] = 3.     # [nS]\n",
    "  pars['VI'] = -80.       # [mV] inhibitory reversal potential\n",
    "  pars['tau_syn_I'] = 5.  # [ms]\n",
    "\n",
    "  pre_spike_train_ex = Poisson_generator(pars, rate=exc_rate, n=80)\n",
    "  pre_spike_train_in = Poisson_generator(pars, rate=inh_rate, n=20)  # 4:1\n",
    "\n",
    "  # Lets first simulate a neuron with identical input but with no spike\n",
    "  # threshold by setting the threshold to a very high value\n",
    "  # so that we can look at the free membrane potential\n",
    "  pars['V_th'] = 1e3\n",
    "  v_fmp, rec_spikes, gE, gI = run_LIF_cond(pars, 0, pre_spike_train_ex,\n",
    "                                           pre_spike_train_in)\n",
    "\n",
    "  # Now simulate a LIP with a regular spike threshold\n",
    "  pars['V_th'] = -55.\n",
    "  v, rec_spikes, gE, gI = run_LIF_cond(pars, 0, pre_spike_train_ex,\n",
    "                                       pre_spike_train_in)\n",
    "  dt, range_t = pars['dt'], pars['range_t']\n",
    "  if rec_spikes.size:\n",
    "      sp_num = (rec_spikes / dt).astype(int) - 1\n",
    "      v[sp_num] = 10  # draw nicer spikes\n",
    "\n",
    "  spike_rate = 1e3 * len(rec_spikes) / pars['T']\n",
    "\n",
    "  cv_isi = 0.\n",
    "  if len(rec_spikes) > 3:\n",
    "    isi = np.diff(rec_spikes)\n",
    "    cv_isi = np.std(isi) / np.mean(isi)\n",
    "\n",
    "  print('\\n')\n",
    "  plt.figure(figsize=(15, 10))\n",
    "  plt.subplot(211)\n",
    "  plt.text(500, -35, f'Spike rate = {spike_rate:.3f} (sp/s), Mean of Free Mem Pot = {np.mean(v_fmp):.3f}',\n",
    "           fontsize=16, fontweight='bold', horizontalalignment='center',\n",
    "           verticalalignment='bottom')\n",
    "  plt.text(500, -38.5, f'CV ISI = {cv_isi:.3f}, STD of Free Mem Pot = {np.std(v_fmp):.3f}',\n",
    "           fontsize=16, fontweight='bold', horizontalalignment='center',\n",
    "           verticalalignment='bottom')\n",
    "\n",
    "  plt.plot(pars['range_t'], v_fmp, 'r', lw=1.,\n",
    "           label='Free mem. pot.', zorder=2)\n",
    "  plt.plot(pars['range_t'], v, 'b', lw=1.,\n",
    "           label='mem. pot with spk thr', zorder=1, alpha=0.7)\n",
    "  plt.axhline(pars['V_th'], 0, 1, color='k', lw=1., ls='--',\n",
    "              label='Spike Threshold', zorder=1)\n",
    "  plt.axhline(np.mean(v_fmp),0, 1, color='r', lw=1., ls='--',\n",
    "              label='Mean Free Mem. Pot.', zorder=1)\n",
    "  plt.ylim(-76, -39)\n",
    "  plt.xlabel('Time (ms)')\n",
    "  plt.ylabel('V (mV)')\n",
    "  plt.legend(loc=[1.02, 0.68])\n",
    "\n",
    "  plt.subplot(223)\n",
    "  plt.plot(pars['range_t'][::3], gE[::3], 'r', lw=1)\n",
    "  plt.xlabel('Time (ms)')\n",
    "  plt.ylabel(r'$g_E$ (nS)')\n",
    "\n",
    "  plt.subplot(224)\n",
    "  plt.plot(pars['range_t'][::3], gI[::3], 'b', lw=1)\n",
    "  plt.xlabel('Time (ms)')\n",
    "  plt.ylabel(r'$g_I$ (nS)')\n",
    "\n",
    "  plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "**Mean-driven and Fluctuation-driven regimes**\n",
    "\n",
    "If we look at the figure above, we note that when the mean FMP is above spike threshold, the fluctuations in the FMP are rather small, and the neuron spikes in a fairly regular fashion. This regime, where the mean FMP is above the spike threshold, is called **mean-driven regime**. \n",
    "\n",
    "\n",
    "When the mean FMP is below the spike threshold, the fluctuations in the FMP are large, and the neuron's spikes are driven by these fluctuations. As a consequence, the neuron spikes in more  Poisson-like fashion. This regime, where the mean FMP is below the spike threshold, and spikes are driven by the fluctuations, is called **fluctuation-driven regime**. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Think!\n",
    "\n",
    "- How much can you increase the spike pattern variability? Under what condition(s) might the neuron respond with Poisson-type spikes? Note that we injected Poisson-type spikes. (Think of the answer in terms of the ratio of the exc. and inh. input spike rates.)\n",
    "\n",
    "- Link to the balance of excitation and inhibition: one of the definitions of excitation and inhibition balance is that mean free membrane potential remains constant as excitatory and inhibitory input rates are increased. What do you think happens to the neuron firing rate as we change excitatory and inhibitory rates while keeping the neuron in balance?  See [Kuhn, Aertsen, and Rotter (2004)](https://www.jneurosci.org/content/jneuro/24/10/2345.full.pdf) for much more on this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "text",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:47.596972Z",
     "iopub.status.busy": "2021-05-25T01:13:47.595775Z",
     "iopub.status.idle": "2021-05-25T01:13:47.598795Z",
     "shell.execute_reply": "2021-05-25T01:13:47.598310Z"
    }
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_RealNeurons/solutions/W2D3_Tutorial3_Solution_264ec1d4.py)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Section 2: Short-term synaptic plasticity\n",
    "Above, we modeled synapses with fixed weights. Now we will explore synapses whose weight change in some input conditions. \n",
    "\n",
    "Short-term plasticity (STP) is a phenomenon in which synaptic efficacy changes over time in a way that reflects the history of presynaptic activity. Two types of STP, with opposite effects on synaptic efficacy, have been experimentally observed. They are known as Short-Term Depression (STD) and Short-Term Facilitation (STF).\n",
    "\n",
    "The mathematical model (_for more information see [here](http://www.scholarpedia.org/article/Short-term_synaptic_plasticity)_) of STP is based on the concept of a limited pool of synaptic resources available for transmission ($R$), such as, for example, the overall amount of synaptic vesicles at the presynaptic terminals. The amount of presynaptic resource changes in a dynamic fashion depending on the recent history of spikes. \n",
    "\n",
    "Following a presynaptic spike, (i) the fraction $u$ (release probability) of the available pool to be utilized increases due to spike-induced calcium influx to the presynaptic terminal, after which (ii) $u$ is consumed to increase the post-synaptic conductance. Between spikes, $u$ decays back to zero with time constant $\\tau_f$ and $R$ recovers to 1 with time constant $\\tau_d$. In summary, the dynamics of excitatory (subscript $E$) STP are given by:\n",
    "\n",
    "\\\\\n",
    "\n",
    "\\begin{eqnarray}\n",
    "&& \\frac{du_E}{dt} &=& -\\frac{u_E}{\\tau_f} + U_0(1-u_E^-)\\delta(t-t_{\\rm sp}) \\\\[.5mm]\n",
    "&& \\frac{dR_E}{dt} &=& \\frac{1-R_E}{\\tau_d} - u_E^+ R_E^- \\delta(t-t_{\\rm sp}) \\qquad (6) \\\\[.5mm]    \n",
    "&& \\frac{dg_E(t)}{dt} &=& -\\frac{g_E}{\\tau_E} + \\bar{g}_E u_E^+ R_E^- \\delta(t-t_{\\rm sp})\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\\\\n",
    "\n",
    "where $U_0$ is a constant determining the increment of $u$ produced by a spike. $u_E^-$ and $R_E^-$ denote the corresponding values just before the spike arrives, whereas $u_E^+$ refers to the moment right after the spike. $\\bar{g}_E$ denotes the maximum excitatory conductane, and $g_E(t)$ is calculated for all spiketimes $k$, and decays over time with a time constant $\\tau_{E}$. Similarly, one can obtain the dynamics of inhibitory STP (i.e., by replacing the subscript $E$ with $I$).\n",
    "\n",
    "\n",
    "The interplay between the dynamics of $u$ and $R$ determines whether the joint effect of $uR$ is dominated by *depression* or *facilitation*. In the parameter regime of $\\tau_d \\gg \\tau_f$  and for large $U_0$, an initial spike incurs a large drop in $R$ that takes a long time to recover; therefore, the synapse is STD-dominated. In the regime of $\\tau_d \\ll \\tau_f$ and for small $U_0$, the synaptic efficacy is increased gradually by spikes, and consequently, the synapse is STF-dominated. This phenomenological model successfully reproduces the kinetic dynamics of depressed and facilitated synapses observed in many cortical areas."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Exercise 2: Compute $du$, $dR$ and $dg$\n",
    "\n",
    "As we learned in several previous tutorials, the Euler numerical integration method involves the calculation of each derivative at step $n$:\n",
    "\n",
    "\\\\\n",
    "\n",
    "\\begin{eqnarray}\n",
    "du_E &=& -\\frac{u_E[t]}{\\tau_f} dt + U_0(1-u_E[t])\\cdot \\text{sp_or_not[t+dt]} \\\\\n",
    "dR_E &=& \\frac{1-R_E[t]}{\\tau_d} dt - u_E[t+dt]R_E[t]\\cdot \\text{sp_or_not[t+dt]} \\\\\n",
    "dg_E &=& -\\frac{g_E[t]}{\\tau_{E}} dt + \\bar{g}_Eu_E[t+dt]R_E[t]\\cdot \\text{sp_or_not[t+dt]} \\\\\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\\\\n",
    "\n",
    "where $\\text{sp_or_not}=1$ if there's a spike in the time window $dt$, and $\\text{sp_or_not}=0$  otherwise. In addition, note that any spike train generated by our `Poisson_generator` is binary. Then, the values are updated:\n",
    "\n",
    "\\\\\n",
    "\n",
    "\\begin{eqnarray}\n",
    "  u_E[t+dt] &=& u_E[t] + du_E \\\\\n",
    "  R_E[t+dt] &=& R_E[t] + dR_E \\\\\n",
    "  g_E[t+dt] &=& g_E[t] + dg_E \\\\\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\\\\n",
    "\n",
    "Similarly, one can obtain the dynamics of inhibitory conductance.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:47.606003Z",
     "iopub.status.busy": "2021-05-25T01:13:47.604781Z",
     "iopub.status.idle": "2021-05-25T01:13:47.606637Z",
     "shell.execute_reply": "2021-05-25T01:13:47.607088Z"
    }
   },
   "outputs": [],
   "source": [
    "def dynamic_syn(g_bar, tau_syn, U0, tau_d, tau_f, pre_spike_train, dt):\n",
    "  \"\"\"\n",
    "  Short-term synaptic plasticity\n",
    "\n",
    "  Args:\n",
    "    g_bar           : synaptic conductance strength\n",
    "    tau_syn         : synaptic time constant [ms]\n",
    "    U0              : synaptic release probability at rest\n",
    "    tau_d           : synaptic depression time constant of x [ms]\n",
    "    tau_f           : synaptic facilitation time constantr of u [ms]\n",
    "    pre_spike_train : total spike train (number) input\n",
    "                      from presynaptic neuron\n",
    "    dt              : time step [ms]\n",
    "\n",
    "  Returns:\n",
    "    u               : usage of releasable neurotransmitter\n",
    "    R               : fraction of synaptic neurotransmitter resources available\n",
    "    g               : postsynaptic conductance\n",
    "\n",
    "  \"\"\"\n",
    "\n",
    "  Lt = len(pre_spike_train)\n",
    "  # Initialize\n",
    "  u = np.zeros(Lt)\n",
    "  R = np.zeros(Lt)\n",
    "  R[0] = 1.\n",
    "  g = np.zeros(Lt)\n",
    "\n",
    "  # simulation\n",
    "  for it in range(Lt - 1):\n",
    "\n",
    "    #########################################################################\n",
    "    ## TODO for students: compute du, dx and dg, remove NotImplementedError #\n",
    "    # Note pre_spike_train[i] is binary, i.e., sp_or_not in the i-th timebin\n",
    "    # Fill out function and remove\n",
    "    raise NotImplementedError(\"Student excercise: compute the STP dynamics\")\n",
    "    #########################################################################\n",
    "    # Compute du\n",
    "    du = ...\n",
    "    u[it + 1] = u[it] + du\n",
    "    # Compute dR\n",
    "    dR = ...\n",
    "    R[it + 1] = R[it] + dR\n",
    "    # Compute dg\n",
    "    dg = ...\n",
    "    g[it + 1] = g[it] + dg\n",
    "\n",
    "  return u, R, g\n",
    "\n",
    "\n",
    "# Uncomment this line after completing the dynamic_syn function\n",
    "# _ = my_illus_STD(Poi_or_reg=False, rate=20., U0=0.5, tau_d=100., tau_f=50.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 430
    },
    "colab_type": "text",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:47.643599Z",
     "iopub.status.busy": "2021-05-25T01:13:47.619372Z",
     "iopub.status.idle": "2021-05-25T01:13:48.198378Z",
     "shell.execute_reply": "2021-05-25T01:13:48.198819Z"
    },
    "outputId": "3486131c-4fd4-4a8c-c199-aa3519ed4330"
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_RealNeurons/solutions/W2D3_Tutorial3_Solution_0bbde439.py)\n",
    "\n",
    "*Example output:*\n",
    "\n",
    "<img alt='Solution hint' align='left' width=853 height=414 src=https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/tutorials/W2D3_RealNeurons/static/W2D3_Tutorial3_Solution_0bbde439_0.png>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Section 2.1: Short-term syaptic depression (STD)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "### Interactive Demo: STD Explorer with input rate\n",
    "Below, an interactive demo that shows how Short-term synaptic depression (STD) changes for different firing rates of the presynaptic spike train and how the amplitude synaptic conductance $g$ changes with every incoming spike until it reaches its stationary state.\n",
    "\n",
    "Does it matter if the neuron fires in a Poisson manner, rather than regularly?\n",
    "\n",
    "**Note:** `Poi_or_Reg=1`: for *Posisson type* and `Poi_or_Reg=0`: for *regular* presynaptic spikes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 494,
     "referenced_widgets": [
      "4bdd0c4d022b4ac0b46e774c2fed082d",
      "86d7df0fac9645869eec5fb938222452",
      "971b2490f88a48db87762ff248571937",
      "e81953fc23c342daa3f24f092e8ebfd1",
      "1a786137e6864f7faef8f702c85c612c",
      "0a3644ffc4054c2086b8bb54f25af302",
      "4e30fa8de89846c7ae1201e1cda02fe8",
      "8f5c9c5453be4457bb3b7b2a7f37f806",
      "ba0589186c1f41b183841429417141b9",
      "af105006e7964e108239bff387ab70a9"
     ]
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:48.207241Z",
     "iopub.status.busy": "2021-05-25T01:13:48.206682Z",
     "iopub.status.idle": "2021-05-25T01:13:48.792581Z",
     "shell.execute_reply": "2021-05-25T01:13:48.792101Z"
    },
    "outputId": "c660d3e0-a72a-45d0-948c-dbd7c0310392"
   },
   "outputs": [],
   "source": [
    "#@title\n",
    "\n",
    "#@markdown Make sure you execute this cell to enable the widget!\n",
    "\n",
    "\n",
    "def my_STD_diff_rate(rate, Poi_or_Reg):\n",
    "      _ = my_illus_STD(Poi_or_reg=Poi_or_Reg, rate=rate)\n",
    "\n",
    "\n",
    "_ = widgets.interact(my_STD_diff_rate, rate=(10., 100.1, 5.),\n",
    "                     Poi_or_Reg=(0, 1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "text",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:48.799440Z",
     "iopub.status.busy": "2021-05-25T01:13:48.798870Z",
     "iopub.status.idle": "2021-05-25T01:13:48.801628Z",
     "shell.execute_reply": "2021-05-25T01:13:48.802085Z"
    }
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_RealNeurons/solutions/W2D3_Tutorial3_Solution_bedc4823.py)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "### Synaptic depression and presynaptic firing rate\n",
    "Once, I asked an experimentalist about the experimental values of the PSP amplitude produced by a connection between two neocortical excitatory neurons. She asked: \"At what frequency?\" I was confused, but you will understand her question, now that you know that PSP amplitude depends on the spike history, and therefore on the spike rate of the presynaptic neuron. \n",
    "\n",
    "Here, we will study how the ratio of the synaptic conductance corresponding to the first and 10th spikes change as a function of the presynaptic firing rate (experimentalists often take the ratio of first and second PSPs). \n",
    "\n",
    "For computational efficiency, we assume that the presynaptic spikes are regular. This assumption means that we do not have to run multiple trials. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 323
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:48.809133Z",
     "iopub.status.busy": "2021-05-25T01:13:48.808554Z",
     "iopub.status.idle": "2021-05-25T01:13:49.374272Z",
     "shell.execute_reply": "2021-05-25T01:13:49.374778Z"
    },
    "outputId": "e7e744f3-da42-4922-9614-c46d0af6eea9"
   },
   "outputs": [],
   "source": [
    "# @markdown STD conductance ratio with different input rate\n",
    "# Regular firing rate\n",
    "input_rate = np.arange(5., 40.1, 5.)\n",
    "g_1 = np.zeros(len(input_rate))  # record the the PSP at 1st spike\n",
    "g_2 = np.zeros(len(input_rate))  # record the the PSP at 10th spike\n",
    "\n",
    "for ii in range(len(input_rate)):\n",
    "   g_1[ii], g_2[ii] = my_illus_STD(Poi_or_reg=False, rate=input_rate[ii],\n",
    "                                   plot_out=False, U0=0.5, tau_d=100., tau_f=50)\n",
    "\n",
    "plt.figure(figsize=(11, 4.5))\n",
    "\n",
    "plt.subplot(121)\n",
    "plt.plot(input_rate, g_1, 'm-o', label='1st Spike')\n",
    "plt.plot(input_rate, g_2, 'c-o', label='10th Spike')\n",
    "plt.xlabel('Rate [Hz]')\n",
    "plt.ylabel('Conductance [nS]')\n",
    "plt.legend()\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.plot(input_rate, g_2 / g_1, 'b-o')\n",
    "plt.xlabel('Rate [Hz]')\n",
    "plt.ylabel(r'Conductance ratio $g_{10}/g_{1}$')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "As we increase the input rate the ratio of the first to tenth spike is increased, because the tenth spike conductance becomes smaller. This is a clear evidence of synaptic depression, as using the same amount of current has a smaller effect on the neuron."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Section 2.2: Short-term synaptic facilitation (STF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "### Interactive Demo: STF explorer with input rate\n",
    "Below, we see an illustration of a short-term facilitation example. Take note of the change in the synaptic variables: `U_0`, `tau_d`, and  `tau_f`.\n",
    "\n",
    "- for STD, `U0=0.5, tau_d=100., tau_f=50.`\n",
    "\n",
    "- for STP, `U0=0.2, tau_d=100., tau_f=750.`\n",
    " \n",
    "How does the synaptic conductance change as we change the input rate? What do you observe in the case of a regular input and a Poisson type one? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 494,
     "referenced_widgets": [
      "ca94a0de3ace4e5ab95b2e63d6cb6d17",
      "3ae7093a862e4083b16a0089aaf6167e",
      "bdafc4ef98624fcb9bed11d92f6aa2f7",
      "b339fcb5ea5e45d0a5c59c09d5d38d0b",
      "daaca0b03c874f51b7409ad4348acd23",
      "e5758bdc1d374caeb7669c2314cb0376",
      "163d7f9eb89749f18723503421094bc3",
      "de5d81fc0aa848f8bc9859799f7653ce",
      "e976d75c46c84e04ab76e408872c3059",
      "c0552375763d4e5f828b1bedeac0030f"
     ]
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:49.404172Z",
     "iopub.status.busy": "2021-05-25T01:13:49.403605Z",
     "iopub.status.idle": "2021-05-25T01:13:49.915867Z",
     "shell.execute_reply": "2021-05-25T01:13:49.915376Z"
    },
    "outputId": "6fbaa0b8-5d4c-42ff-e295-80651696a9f8"
   },
   "outputs": [],
   "source": [
    "# @title\n",
    "\n",
    "# @markdown Make sure you execute this cell to enable the widget!\n",
    "\n",
    "\n",
    "def my_STD_diff_rate(rate, Poi_or_Reg):\n",
    "  _ = my_illus_STD(Poi_or_reg=Poi_or_Reg, rate=rate, U0=0.2, tau_d=100., tau_f=750.)\n",
    "\n",
    "\n",
    "_ = widgets.interact(my_STD_diff_rate, rate=(4., 40.1, 2.),\n",
    "                     Poi_or_Reg=(0, 1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "### Synaptic facilitation and presynaptic firing rate\n",
    "\n",
    "Here, we will study how the ratio of the synaptic conductance corresponding to the $1^{st}$ and $10^{th}$ spike changes as a function of the presynaptic rate. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 322
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:49.924844Z",
     "iopub.status.busy": "2021-05-25T01:13:49.924292Z",
     "iopub.status.idle": "2021-05-25T01:13:50.837732Z",
     "shell.execute_reply": "2021-05-25T01:13:50.837261Z"
    },
    "outputId": "aa9fc5d6-a27c-48e3-ef03-872352f6a5f2"
   },
   "outputs": [],
   "source": [
    "# @title STF conductance ratio with different input rates\n",
    "# Regular firing rate\n",
    "input_rate = np.arange(2., 40.1, 2.)\n",
    "g_1 = np.zeros(len(input_rate))  # record the the PSP at 1st spike\n",
    "g_2 = np.zeros(len(input_rate))  # record the the PSP at 10th spike\n",
    "\n",
    "for ii in range(len(input_rate)):\n",
    "   g_1[ii], g_2[ii] = my_illus_STD(Poi_or_reg=False, rate=input_rate[ii],\n",
    "                                   plot_out=False,\n",
    "                                   U0=0.2, tau_d=100., tau_f=750.)\n",
    "\n",
    "plt.figure(figsize=(11, 4.5))\n",
    "\n",
    "plt.subplot(121)\n",
    "plt.plot(input_rate, g_1, 'm-o', label='1st Spike')\n",
    "plt.plot(input_rate, g_2, 'c-o', label='10th Spike')\n",
    "plt.xlabel('Rate [Hz]')\n",
    "plt.ylabel('Conductance [nS]')\n",
    "plt.legend()\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.plot(input_rate, g_2 / g_1, 'b-o',)\n",
    "plt.xlabel('Rate [Hz]')\n",
    "plt.ylabel(r'Conductance ratio $g_{10}/g_{1}$')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Think!\n",
    "\n",
    "Why does the ratio of the first and tenth spike conductance changes in a non-monotonic fashion for synapses with STF, even though it decreases monotonically for synapses with STD?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "text",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:50.842473Z",
     "iopub.status.busy": "2021-05-25T01:13:50.841930Z",
     "iopub.status.idle": "2021-05-25T01:13:50.844617Z",
     "shell.execute_reply": "2021-05-25T01:13:50.845056Z"
    }
   },
   "source": [
    "[*Click for solution*](https://github.com/NeuromatchAcademy/course-content/tree/master//tutorials/W2D3_RealNeurons/solutions/W2D3_Tutorial3_Solution_f301e996.py)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Summary\n",
    "\n",
    "Congratulations! You have just finished the last tutorial of this day. Here, we saw how to model conductance-based synapses and also how to incorporate short-term dynamics in synaptic weights. \n",
    "\n",
    "We covered the:\n",
    "\n",
    "- static synapses and how excitation and inhibition affect the neuronal output\n",
    "- mean- or fluctuation-driven regimes\n",
    "- short-term dynamics of synapses (both facilitation and depression)\n",
    "\n",
    "Finally, we incorporated all the aforementioned tools to study how a change in presynaptic firing history affects the synaptic weights!\n",
    "\n",
    "There are many interesting things that you can try on your own to develop a deeper understanding of biological synapses. A couple of those are mentioned below in the optional boxes -- if you have time.\n",
    "\n",
    "But now it is time to explore another important feature of biological synapses, i.e., spike timing dependent synaptic plasticity (go to the next tutorial)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Bonus 1: Conductance-based LIF with STP\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "Previously, we looked only at how the presynaptic firing rate affects the presynaptic resource availability and thereby the synaptic conductance. It is straightforward to imagine that, while the synaptic conductances are changing, the output of the postsynaptic neuron will change as well. \n",
    "\n",
    "So, let's put the STP on synapses impinging on an LIF neuron and see what happens. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 354
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:50.859882Z",
     "iopub.status.busy": "2021-05-25T01:13:50.852493Z",
     "iopub.status.idle": "2021-05-25T01:13:50.861980Z",
     "shell.execute_reply": "2021-05-25T01:13:50.862413Z"
    },
    "outputId": "56a7ef0e-2927-4308-dfe0-17275653afff"
   },
   "outputs": [],
   "source": [
    "# @title Function for conductance-based LIF neuron with STP-synapses\n",
    "\n",
    "def run_LIF_cond_STP(pars, I_inj, pre_spike_train_ex, pre_spike_train_in):\n",
    "  \"\"\"\n",
    "  conductance-based LIF dynamics\n",
    "\n",
    "  Args:\n",
    "    pars               : parameter dictionary\n",
    "    I_inj              : injected current [pA]\n",
    "                         The injected current here can be a value or an array\n",
    "    pre_spike_train_ex : spike train input from presynaptic excitatory neuron (binary)\n",
    "    pre_spike_train_in : spike train input from presynaptic inhibitory neuron (binary)\n",
    "\n",
    "  Returns:\n",
    "    rec_spikes : spike times\n",
    "    rec_v      : mebrane potential\n",
    "    gE         : postsynaptic excitatory conductance\n",
    "    gI         : postsynaptic inhibitory conductance\n",
    "\n",
    "  \"\"\"\n",
    "\n",
    "  # Retrieve parameters\n",
    "  V_th, V_reset = pars['V_th'], pars['V_reset']\n",
    "  tau_m, g_L = pars['tau_m'], pars['g_L']\n",
    "  V_init, V_L = pars['V_init'], pars['E_L']\n",
    "  gE_bar, gI_bar = pars['gE_bar'], pars['gI_bar']\n",
    "  U0E, tau_dE, tau_fE = pars['U0_E'], pars['tau_d_E'], pars['tau_f_E']\n",
    "  U0I, tau_dI, tau_fI = pars['U0_I'], pars['tau_d_I'], pars['tau_f_I']\n",
    "  VE, VI = pars['VE'], pars['VI']\n",
    "  tau_syn_E, tau_syn_I = pars['tau_syn_E'], pars['tau_syn_I']\n",
    "  tref = pars['tref']\n",
    "\n",
    "  dt, range_t = pars['dt'], pars['range_t']\n",
    "  Lt = range_t.size\n",
    "\n",
    "  nE = pre_spike_train_ex.shape[0]\n",
    "  nI = pre_spike_train_in.shape[0]\n",
    "\n",
    "  # compute conductance Excitatory synapses\n",
    "  uE = np.zeros((nE, Lt))\n",
    "  RE = np.zeros((nE, Lt))\n",
    "  gE = np.zeros((nE, Lt))\n",
    "  for ie in range(nE):\n",
    "    u, R, g = dynamic_syn(gE_bar, tau_syn_E, U0E, tau_dE, tau_fE,\n",
    "                          pre_spike_train_ex[ie, :], dt)\n",
    "\n",
    "    uE[ie, :], RE[ie, :], gE[ie, :] = u, R, g\n",
    "\n",
    "  gE_total = gE.sum(axis=0)\n",
    "\n",
    "  # compute conductance Inhibitory synapses\n",
    "  uI = np.zeros((nI, Lt))\n",
    "  RI = np.zeros((nI, Lt))\n",
    "  gI = np.zeros((nI, Lt))\n",
    "  for ii in range(nI):\n",
    "    u, R, g = dynamic_syn(gI_bar, tau_syn_I, U0I, tau_dI, tau_fI,\n",
    "                          pre_spike_train_in[ii, :], dt)\n",
    "\n",
    "    uI[ii, :], RI[ii, :], gI[ii, :] = u, R, g\n",
    "\n",
    "  gI_total = gI.sum(axis=0)\n",
    "\n",
    "  # Initialize\n",
    "  v = np.zeros(Lt)\n",
    "  v[0] = V_init\n",
    "  Iinj = I_inj * np.ones(Lt)  # ensure I has length Lt\n",
    "\n",
    "  # simulation\n",
    "  rec_spikes = []  # recording spike times\n",
    "  tr = 0.\n",
    "  for it in range(Lt - 1):\n",
    "    if tr > 0:\n",
    "      v[it] = V_reset\n",
    "      tr = tr - 1\n",
    "    elif v[it] >= V_th:  # reset voltage and record spike event\n",
    "      rec_spikes.append(it)\n",
    "      v[it] = V_reset\n",
    "      tr = tref / dt\n",
    "\n",
    "    # calculate the increment of the membrane potential\n",
    "    dv = (dt / tau_m) * (-(v[it] - V_L) \\\n",
    "                         - (gE_total[it + 1] / g_L) * (v[it] - VE) \\\n",
    "                         - (gI_total[it + 1] / g_L) * (v[it] - VI) + Iinj[it] / g_L)\n",
    "\n",
    "    # update membrane potential\n",
    "    v[it+1] = v[it] + dv\n",
    "\n",
    "  rec_spikes = np.array(rec_spikes) * dt\n",
    "\n",
    "  return v, rec_spikes, uE, RE, gE, RI, RI, gI\n",
    "\n",
    "\n",
    "print(help(run_LIF_cond_STP))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Simulation of a postsynaptic neuron with STP synapses driven by Poisson type spike trains\n",
    "\n",
    "Here we have assumed that both excitatory and inhibitory synapses show short-term depression. Change the nature of synapses and study how spike pattern variability changes.\n",
    "In the interactive demo, `tau_d = 500*tau_ratio (ms)` and  `tau_f = 300*tau_ratio (ms)`.\n",
    "\n",
    "You should compare the output of this neuron with what you observed in the previous tutorial when synapses were assumed to be static. \n",
    "\n",
    "_Note: it will take slightly longer time to run each case_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "### Interactive Demo: LIF with STP Explorer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 534,
     "referenced_widgets": [
      "24245c9a8a554921b289ad71bbbad6a2",
      "11b69eba6b6e4b9983467e9356d30898",
      "9476124f249f41579f47d4f4cf835c74",
      "ed649931930b44ba9221741033571216",
      "0171f55f9e2b4a68b8e30ee82f62426e",
      "190eb9c6bc4b4ebcb7738b1a7949fb5d",
      "e3bc62f3b18f408799c057783e62710b"
     ]
    },
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2021-05-25T01:13:50.890654Z",
     "iopub.status.busy": "2021-05-25T01:13:50.888834Z",
     "iopub.status.idle": "2021-05-25T01:13:54.523491Z",
     "shell.execute_reply": "2021-05-25T01:13:54.523020Z"
    },
    "outputId": "bdd956b9-1001-4074-fdcb-1712059b0fd9"
   },
   "outputs": [],
   "source": [
    "# @title\n",
    "\n",
    "# @markdown Make sure you execute this cell to enable the widget!\n",
    "\n",
    "\n",
    "def LIF_STP(tau_ratio):\n",
    "  pars = default_pars(T=1000)\n",
    "  pars['gE_bar'] = 1.2 * 4  # [nS]\n",
    "  pars['VE'] = 0.           # [mV]\n",
    "  pars['tau_syn_E'] = 5.    # [ms]\n",
    "  pars['gI_bar'] = 1.6 * 4  # [nS]\n",
    "  pars['VI'] = -80.         # [ms]\n",
    "  pars['tau_syn_I'] = 10.   # [ms]\n",
    "\n",
    "  # here we assume that both Exc and Inh synapses have synaptic depression\n",
    "  pars['U0_E'] = 0.45\n",
    "  pars['tau_d_E'] = 500. * tau_ratio  # [ms]\n",
    "  pars['tau_f_E'] = 300. * tau_ratio  # [ms]\n",
    "\n",
    "  pars['U0_I'] = 0.45\n",
    "  pars['tau_d_I'] = 500. * tau_ratio  # [ms]\n",
    "  pars['tau_f_I'] = 300. * tau_ratio  # [ms]\n",
    "\n",
    "  pre_spike_train_ex = Poisson_generator(pars, rate=15, n=80)\n",
    "  pre_spike_train_in = Poisson_generator(pars, rate=15, n=20)  # 4:1\n",
    "\n",
    "  v, rec_spikes, uE, RE, gE, uI, RI, gI = run_LIF_cond_STP(pars, 0,\n",
    "                                                           pre_spike_train_ex,\n",
    "                                                           pre_spike_train_in)\n",
    "\n",
    "  t_plot_range = pars['range_t'] > 200\n",
    "\n",
    "  plt.figure(figsize=(11, 7))\n",
    "  plt.subplot(211)\n",
    "  plot_volt_trace(pars, v, rec_spikes)\n",
    "\n",
    "  plt.subplot(223)\n",
    "  plt.plot(pars['range_t'][t_plot_range], gE.sum(axis=0)[t_plot_range], 'r')\n",
    "  plt.xlabel('Time (ms)')\n",
    "  plt.ylabel(r'$g_E$ (nS)')\n",
    "\n",
    "  plt.subplot(224)\n",
    "  plt.plot(pars['range_t'][t_plot_range], gI.sum(axis=0)[t_plot_range], 'b')\n",
    "  plt.xlabel('Time (ms)')\n",
    "  plt.ylabel(r'$g_I$ (nS)')\n",
    "\n",
    "  plt.tight_layout()\n",
    "\n",
    "\n",
    "_ = widgets.interact(LIF_STP, tau_ratio=(0.2, 1.1, 0.2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "When we vary the tau_ratio we are increasing `tau_f` and `tau_d` i.e. by increasing `tau_ratio` we are increasing the synaptic depression. The effect is same on both Exc and Inh conductances.\n",
    "This is visible as a clear decrease in the firing rate of the neuron from 300-400ms onwards.\n",
    "\n",
    "Not much happens in the beginning because synaptic depression takes some time to become visible.\n",
    "\n",
    "It is curious that while both excitatory and inhibitory conductances have depressed but output firing rate has still decreased.\n",
    "\n",
    "There are two explanations of this:\n",
    "1. excitation has depressed more than the inhibition from their starting values.\n",
    "2. because synaptic conductances have depressed, membrane fluctuation size has decreased.\n",
    "\n",
    "Which is more likely reason? Think."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "---\n",
    "# Bonus 2: STP Synapse Parameter Exploration\n",
    "\n",
    "Vary the parameters of the above simulation and observe the spiking pattern of the postsynaptic neuron. \n",
    "Will the neuron show higher irregularity if the synapses have STP? If yes, what should be the nature of STP on static and dynamic synapses, respectively? \n",
    "\n",
    "Calculate the $CV_{\\rm ISI}$ for different `tau_ratio` after simulating the LIF neuron with STP (Hint:`run_LIF_cond_STP` help you understand the irregularity).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "## Functional implications of short-term dynamics of synapses\n",
    "As you have seen above, if the firing rate is stationary, the synaptic conductance quickly reaches a fixed point. On the other hand, if the firing rate transiently changes, synaptic conductance will vary -- even if the change is as short as a single inter-spike-interval. Such small changes can be observed in a single neuron when input spikes are regular and periodic. If the input spikes are Poissonian, then one may have to perform an average over several neurons.\n",
    "\n",
    "_Come up with other functions that short-term dynamics of synapses can be used to implement and implement them._"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "W2D3_Tutorial3",
   "provenance": [],
   "toc_visible": true
  },
  "kernel": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
